docs/refactor: document data-layer contracts and dedupe token parsing (D1) - #41
Merged
Merged
Conversation
Add docs/data/data_layer_contracts.md stating the boundary invariants: - data/cache is the reusable raw endpoint cache / source of truth; - data/store/PanelStore is a per-run canonical panel artifact, NOT the raw data lake; - durable cache rows stay raw only; front_adjust, PIT index membership, PIT SW industry, ann_date financial alignment, raw price-limit checks, factors/alpha/portfolio/runtime math all remain downstream; - data-update warms caches only and never runs factors/backtests; - data-quality checks and concurrency are later phases (NOT claimed implemented in D1).
TushareFeed and IndexConstituentsFeed each carried their own copy of the external-config JSON parsing (and IndexConstituentsFeed imported a private _lookup_dotted from tushare_feed). Route both onto data/feed/secret.read_token, matching the four sibling feeds (flags/fina/intraday/covariates). - tushare_feed: drop the duplicated module-level _lookup_dotted and the inline json/Path parsing; _read_token is now a one-line wrapper over read_token(self._secret_file, self._token_key). - index_feed: stop importing _lookup_dotted from tushare_feed; _client reads the token via read_token directly. Public constructor signatures, repr() token safety, and fetch/cache behavior are unchanged. Add two narrow tests exercising IndexConstituentsFeed's real _client() token path with a fake tushare.pro_api (token sourced from a fake external config, never leaked; missing-key error stays readable).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Data Layer D1 — the first low-risk data-layer cleanup. Two things only:
cachevsstoreboundary explicit;TushareFeedandIndexConstituentsFeedby routing both onto the existingdata/feed/secret.py::read_token(the four sibling feeds already use it).Behavior-preserving: no data-fetch, cache, factor, alpha, portfolio, runtime,
or report-metric behavior changes.
Changed files (4)
docs/data/data_layer_contracts.md— states:data/cacheis thereusable raw endpoint cache / source of truth;
data/store/PanelStoreis aper-run canonical panel artifact (not the raw lake); durable cache rows stay
raw only;
front_adjust/ PIT index membership / PIT SW industry /ann_datefinancial alignment / raw price-limit checks / factors/alpha/portfolio/runtime
math all remain downstream;
data-updatewarms caches only; data-quality andconcurrency are later phases (not claimed implemented).
data/feed/tushare_feed.py— drop the duplicated module-level_lookup_dottedand the inlinejson/Pathparsing;_read_tokenis now aone-line wrapper over
read_token(self._secret_file, self._token_key).data/feed/index_feed.py— stop importing the private_lookup_dottedfrom
tushare_feed.py;_clientreads the token viaread_tokendirectly.tests/test_index_feed.py— +2 narrow tests exercising the indexfeed's real
_client()token path with a faketushare.pro_api(token sourcedfrom a fake external config, never leaked; missing-key error stays readable).
No test reads the real
.config.json; no test hits the network._lookup_dottedis now fully removed repo-wide; all six tushare-backed feeds(feed/index/flags/fina/intraday/covariates) source the token through
secret.read_token.Verification (rebased onto current
main)pytest -p no:cacheprovider→ 579 passed (current main baseline 577 + 2 newindex-feed tests; rose from Codex's earlier 540 only because the branch now sits
on newer
main, not because D1 changed).ruff check .→ All checks passed!validate-configover all 17config/*.yaml→ all OK.run-phase0 --config config/example.yaml→ic_mean=0.9600, annual_return=0.8408(anchor unchanged — no behavior drift).git diff --check main...HEAD→ clean; merge-marker scan → 0..config.jsontoken value 0 hits;FAKE_TUSHARE_TOKEN…only in the test fixture;.config.json/tushare.tokenonly as config-key/scan-target descriptions.
Explicit non-goals (deferred to D2+)
TushareCacheinternal split;CoverageLedgerstorage change;PanelStoreappend/partition feature;